DOCUMENT RESUME 



ED 401 900 



IR 056 143 



AUTHOR 

TITLE 



INSTITUTION 

PUB DATE 
NOTE 



AVAILABLE FROM 



PUB TYPE 



Bertot, John Carlo; McClure, Charles R. 

Sailor: Maryland's Online Public Information Network. 
Sailor Network Assessment Final Report: Findings and 
Future Sailor Network Development. 

Maryland State Dept, of Education, Baltimore. Div. of 
Library Development and Services. 

1 Sep 96 

1 1 6p . ; For companion report containing detailed data 
collection and study findings, "Sailor Network 
Assessment Final Report Compendium," see IR 056 
144. 

Division of Library and Development Services, 

Maryland State Department of Education, 200 W. 
Baltimore St., Baltimore, MD 21201-2595. 

Reports - Research/Technical (143) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC05 Plus Postage. 

Community; Computer Mediated Communication; *Computer 
Networks; Futures (of Society); Information Networks; 
^Internet; Measurement; Online Systems; Program 
Evaluation; ^Public Libraries; State Government; 
*State Programs; User Needs (information); Use 
Studies 

Maryland; *Sailor Online Public Information Network 
MD 



ABSTRACT 

This report describes the results of an assessment of 
Sailor, Maryland’s Online Public Information Network, which provides 
statewide Internet connection to 100% of Maryland public libraries. 
The concept of a "statewide networked environment" includes 
information services, products, hardware and software, 
telecommunications infrastructure, and resources that users and 
providers of the statewide network receive via the network. The 
statewide networked environment comprises both state networks and 
locally developed information services, and serves as a gateway to 
national and international information services. The purpose of the 
assessment was to: (1) evaluate the value of the Sailor network to 

such target communities as the public library, K~12, business, 
minority, and state and local government communities; (2) recommend 
refinements and modifications to the Sailor network to meet the needs 
of the target communities; (3) measure the nature and extent of 
target community use of the Sailor network; and (4) create 
measurement devices that the Division of Library Development and 
Services (DLDS) can use in the future to assess the Sailor network. 
This report details key findings, future Sailor development models, 
and provides recommendations. Appendices include: Seymour Plan Vision 
Statement; Sailor Network Funding Expenditures Summary and Governance 
Board Membership; The Piper Letter: Making Sense of Web Usage 
Statistics; and Sailor Operations Center Fiscal Year 1996 Annual 
Report. (Contains 12 references.) (SWC) 
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INTRODUCTION 

The Division of Library Development and Services (DLDS) contracted with the 
study team of John Carlo Bertot, Charles R. McClure, and Suzanne Eastham in the 
fall of 1995 to assess the Sailor network. The purpose of the assessment was to: 

• Evaluate the value of the Sailor network to such target communities as the 
public library, K-12, business, minority, and state and local government 
communities; 

• Recommend refinements and modifications to the Sailor network to meet the 
needs of the target communities; 

• Measure the nature and extent of target community use of the Sailor network, 
with emphasis on the incorporation of Sailor network-based resources into 
community activities; and, 

• Create measurement devices that DLDS can use in the future to maintain 
ongoing or periodic assessment of the Sailor network. 

The assessment, therefore, focused on the present use and future directions of Sailor. 

This final report is a summary version of the study findings. The final report 
Compendium contains the detailed study findings, study data collection activity 
write-ups, detailed methodologies, data collection tools, and consultant notes on the 
uses of. the study’s data collection instruments. Readers interested in the details of 
the study’s data collection activities should consult the final report Compendium. 
Together, the final report and the final report compendium demonstrate two different 
purposes: (1) the final report presents specific issues and findings pertaining to the 
Sailor network; and, (2) the final report Compendium presents tools for DLDS and 
Sailor-connected organizations to perform continued Sailor evaluation activities, but 
may also serve to provide other statewide electronic network evaluators with a set of 
tools to assist them in their state network evaluation activities. 

The Statewide Electronic Network Environment 

The notion of the "statewide networked environment" encompasses a range of 
electronic networked activities and services. Minimally, the statewide networked 
environment includes information services, products, hardware and software, 
telecommunications infrastructure, and resources that users and providers of the 
statewide network receive via the network. In this environment, state networks 
provide information services, although locally developed information services (e.g., 
local government, school districts, public libraries) also comprise the statewide 
networked environment. In addition, such networks serve as gateways to national 
and international information services. Both the networking of users and resources 
within the state as well the connecting of these users to other persons and resources 
outside of the state are considered part of this environment. 
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The concept of networked information services is an evolving one. Such services 
can be offered by individuals, libraries, computer centers, publishers, networks, 
government agencies, or a host of other organizations and groups with access to the 
Internet and the evolving National Information Infrastructure (Nil) (Huth and Gould, 
1994) and the Global Information Infrastructure (GII) (Gore, 1995). Networked 
information services comprise bulletin boards; e-mail; listservs; remote access to 
distant databases, software, and high speed computing; and, can support 
collaborative efforts among geographically dispersed individuals-to name but a few 
network-enabling capabilities. Key aspects of "networked information services" are 
that: 



• There are numerous information providers connected by a statewide network 
at the state and local levels. 

• There are a range of electronic information services available to users. 

• Access to and use of these services continues to increase. 

• Technological advances create an extremely fluid and rapidly changing 
networked environment. 

Despite the fact that many state governments have built significant networks 
and are connected to the Internet and the evolving GII and Nil, there is little 
knowledge of how such connectivity affects the state in such socio-economic terms as 
enhanced citizen productivity, education, employment, and creating a more cost 
effective and efficient exchange between government and citizens, to name a few key 
issue areas. 

There is evidence that a restructuring of the computing and communications 
infrastructure as a result of the availability and use of electronic information is 
occurring and that this will have a fundamental impact on government institutions. 
Already, this restructuring is affecting the communication customs and expectations 
of researchers in a variety of fields. In a larger sense, this restructuring is affecting 
the entire information transfer cycle from the creation, structuring, and 
representation of information to its dissemination and use by the members of various 
communities (McClure, 1993). 

Traditional criteria used in assessing information services, e.g., extensiveness, 
efficiency, effectiveness, impact, service quality, and usefulness, may serve as a 
beginning model for networked information services assessment. Wor king with these, 
and other criteria, McClure and Lopata (1996) have developed a manual that 
describes qualitative and quantitative techniques - as well as offering a example user 
survey - for assessing an academic networked environment. In addition, CAUSE, an 
association for managing and using information resources in higher education, 
published an excellent tool, Self-Assessment for Campus Information Technology 
Services (Fleit, 1994) as well as Evaluation Guidelines for Institutional Information 
Resources (CAUSE, 1995). These assessment tools can serve as a beginning point for 
those in the process of assessing the statewide networked environment. 
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The Maryland Statewide Network Environment 

The state of Maryland is on the forefront of providing statewide public access to 
the Internet. Since the beginning implementation of the Sailor network in 1993, 
DLDS began the innovative, challenging, and nationally unprecedented task of 
connecting the entire state to the Internet through public libraries (McClure, Bertot, 
and Beachboard, 1995). The goal to provide free, local, and instant access to 
electronic networked information resources pushed Maryland to the forefront of 
public librarianship. 

Today, Maryland is far ahead of the rest of the nation in terms of public library 
Internet connectivity. At present, 44.6% of the nation’s public libraries are connected 
to the Internet in some way (Bertot, McClure, and Zweizig, 1996). Regionally, 39.0% 
of public libraries in the South (a census-based category into which Maryland falls) 
have Internet connections. In Maryland, 100.0% of public libraries now have an 
Internet connection through the Sailor network. 

Through creativity, DLDS-public library collaboration, and an entrepreneurial 
spirit, Maryland is now in a position that far exceeds the current state of the nation in 
terms of Internet connectivity. Indeed, while the rest of the nation’s public libraries 
contend with issues of establishing Internet connections, Maryland, through the 
Sailor network, is entering a new era of electronic content and service development. 
This will once again place Maryland on the frontier of statewide electronic networking. 

With the completion of the Sailor network backbone, and prior to moving fully into 
electronic content and service development, DLDS recognized the need for an 
evaluation of the Sailor network. Such an evaluation would serve to guide the future 
development of the Sailor network as Sailor enters the era of content and service 
initiatives, and provide a basis for future Sailor network activity measurement and 
evaluation. 

Measurement and Evaluation 

Performance measures represent a broad managerial/evaluation concept that 
encompasses measurement of inputs (indicators of the resources essential to provide 
a service), outputs (indicators of the services resulting from the use of those 
resources), and impacts (the affect of these outcomes on other variables or factors). 
They are an essential means to assess the statewide networked environment. 
Performance measures serve a number of useful purposes. They can: 

• Identify those aspects of the network that are successful versus those aspects 
that are less successful. 

• Provide trend-data to assess changes in the network and network services over 
time. 

• Assist decision makers to allocate or reallocate resources needed to plan for 
future network development. 



O 

ERIC 



3 



Bertot and McClure 



Sailor Network Assessment Final Report 



September 1. 1996 



• Monitor network activities and services to inform managers of any changes in 
activities or the quality of services. 

• Determine the degree to which users are satisfied with the network and 
network services. 

• Assist network managers to justify expenditures and be accountable for those 
expenditures. 

Performance measures, then, ask decision makers to answer the question: How well 
is the service or activity doing what it claims to be doing? 

Performance measures can also assist managers to formally evaluate the 
network. Thus, evaluation is the process of identifying and collecting data about 
specific services or activities, establishing criteria to assess their success, and 
determining the degree to which the service or activity accomplishes stated 
objectives. Evaluation, therefore, is a decision making tool intended primarily to 
assist decision makers allocate resources that best accomplish organizational goals. 
Evaluation reflects value judgements on the part of the evaluator regarding the 
adequacy, appropriateness, and success of a particular service or activity. 

In a broader organizational context, measurement and evaluation of networked 
information services are essential for resource allocation, planning, and improving 
services. Without measures that can evaluate particular services, decision makers 
must rely on intuition and anecdotal information as a basis for assessing the 
usefulness and value of a particular service. Perhaps most importantly, 
measurement and evaluation provide feedback opportunities for users to make 
known the extent to which those services meet their needs. 

Approaches for evaluating networked information services can be based on the 
following criteria: 

• Extensiveness: how much of the service has been provided, e.g., number of 
users logging-in per week on a bulletin board, or the number of participants of a 
particular listserv. 

• Efficiency: the use of resources in providing or accessing networked 

information services, e.g., cost per session in providing access to remote users 
of an on-line catalog, or average time required to successfully telnet to a 
remote database. 

• Effectiveness: how well the networked information service met the objectives 
of the provider or the user, e.g., success rate of identifying and accessing the 
information needed by the user. 

• Service quality : how well a service or activity is done, e.g., percentage of 
transactions where users acquire the information they need. 
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• Impact : how a service made a difference in some other activity or situation, 
e.g., the degree to which network users enhanced their ability to gain 
employment or pursue business. 

• Usefulness : this is a measure of appropriateness, that is, the degree to which 
the services are useful or appropriate for the individual user, e.g., percentage of 
services of interest to different types of user audiences. 

Although evaluations of networked information services need to consider 
extensiveness and efficiency criteria, this study focused primarily on effectiveness, 
service quality, usefulness, and impact measures. Developing measures of impacts 
from networked services, however, remains a very difficult task. 

Providers of networked information services must not accept as a "given" that 
their services, resources, and technical procedures are efficient and effective. 
Ongoing evaluation activities are essential to support the provider's planning process. 
Planning and evaluation are two sides of the same coin. Each will be more successful 
when the other is part of the overall services design and implementation approach. 

Developing, operationalizing, and validating a range of performance measures that 
encourage electronic network service providers to assess what types of networked 
information services have what level of quality, impacts on the network users, and 
costs is essential if administrators of networked information in the electronic network 
setting are to justify such services and better meet user information needs. 



STUDY METHODOLOGY 

The assessment team engaged in a number of data collection activities between 
October 1, 1995 and June 1, 1996 to derive the findings presented in this report. 
These activities included: 

• Case studies of the five 1 Maryland library systems that included focus groups 
and interviews with site 

- Public library administrators and librarians, 

- Local and state government officials, 

- Higher education faculty and administrators, 

K-12 teachers and administrators, 

Local business leaders, 

- Members of the press/media, and 

- Internet Service Providers (ISPs); 



!The initial Sailor assessment plan included a four site demographically- and 
regionally-based case study approach that included Baltimore City, Harford County, 
Montgomery County, and Wicomico County. The assessment team added Garrett 
County to the original case site selection to gain a Western Maryland perspective on 
Sailor. 
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• Mail and quick response surveys with 

School library media specialists in selected case sites, 

Sailor Master Trainers, and 

The Maryland Association of Public Library Administrators (MAPLA); 

• Focus groups and interviews with key Sailor entities, including 

- DLDS, 

- Sailor Operations, 

- Network Navigators, 

- Network Coordination Council, and 

- MAPLA; 

• Preliminary study result presentations with the Advisory Committee and at 
the May 8, 1996 Maryland Library Association annual meeting; 

• Sailor network traffic data collection for the month of March, 1996, including 

- Overall Sailor usage, 

- Help desk transactions, and 

- Case site Ascend router traffic;2 

• Sailor Web page assessment in comparison to five other statewide state 
library-run network Web pages; 

• Content analysis of case site, DLDS, and Sailor Operations Sailor-related 
planning, training, and manual documentation; and, 

• Pre-testing data collection instruments, continual consultation with, and 
presentations to the Sailor network assessment Advisory Committee, DLDS, 
and the Maryland Library Association. 

Together, these data collection activities provided the assessment team with the 
ability to identify both general and specific Sailor network operation, management, 
and technical issues (see Figure 1). In all, the study team conducted 10 focus groups 
and 23 in-person and/or telephone interviews that included over 125 individuals from 
the Sailor-related communities identified above (see Figure 2). Specific details of each 
data collection activity is available in the final report Compendium. 

Throughout the data collection component of this study, a number of factors 
affected the ability of the study team to collect assessment data. Several of these 
factors are identified below. 



2 Due to technical difficulties during March 1996, Ascend router data was 
collected between May 20, 1996 and July 25, 1996. 
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Figure 1. Methodologies for Developing Assessment Techniques. 



Qualitative 


Function/Purpose 


Focus Groups 


Explore identified key issue areas of network 
content, services, governance, and 
performance. 

Use findings to inform broader quantitative 
data collection activities such as mail and 
electronic surveys. 


Small Group and Individual 
Interviews 


In-depth exploration of network content, 
services, governance and performance with 
key state and local government policy makers. 

Assess the relationship between components 
of the network and future government use 
and development of networked resources. 

Use findings to inform broader quantitative 
data collection activities such sis mail and 
electronic surveys. 


Case Sites 


In-depth exploration of selected state 
communities (i.e., counties) and target 
audiences within those communities (e.g, k-12, 
public libraries) use of and involvement with 
the statewide network. 

Use findings to inform broader quantitative 
data collection activities such as mail and 
electronic surveys. 


Content Analysis 


Gather various documentation and reports to 
review historical development and evolution 
of electronic network activities. 


Quantitative 


Function/Purpose 


Mail/Electronic Surveys 


Further explore identified key issue areas of 
network content, services, governance, and 
performance with broader state population. 

Test findings from qualitative data collection 
activities with broader state population. 


Network Traffic Measures 


Collect network traffic use statistics such as 
users, user access points, information and 
service content use, and network server and 
router load. 
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Figure 2. Sailor Evaluation Study Methodology. 
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Measurement Issues 

Due to the complexity and multi-dimensionality of the electronic networked 
environment, evaluators face numerous methodological issues. For example, using 
the definition of a statewide network put forth later in this report, a researcher must 
decide which component of the network he/she is attempting to measure, what 
methods are appropriate for data collection activities within that network 
component, what variables are to measure, how best to define and measure those 
variables, and what data analysis tools are most appropriate to allow for meaningful 
interpretation of the collected data. 

In addition to methodological issues, statewide network evaluation activities occur 
within a rapidly changing technological environment. While technological advances 
serve to enhance electronic network services, continual changes in technology force 
network evaluators to continually revise their network measures. In essence, this 
creates a situation in which evaluators are continually using modified and/or new 
measures, and are thus never really able to develop valid and reliable measurement 
tools through replication. 

The methodological and technological aspects of electronic network evaluation 
create a challenging environment for network evaluators. Below are selected 
measurement issues that reside within this environment. 

Distributed Network Environment 

The degree to which a statewide network is distributed significantly affects the 
development of network performance measures. A distributed network environment, 
while fostering flexibility, presents the following challenges: 

• Oftentimes there is no single person or unit that is "in charge" of networking 
development and services. As such, obtaining data about specific networking 
services or activities requires the evaluator to work with a number of 
individuals or units. 

• Distributed systems, with different system administrators, and with varying 
levels of interest in evaluation, may have different types of data that measure 
very different aspects of the network. 

• Evaluation techniques and measures that are needed and are appropriate for 
network-wide assessment may be different than those needed by individual 
network system components. 

Thus, the distributed network environment may complicate the process for ongoing 
evaluation and performance assessment. 

Management Information Systems (MIS) 

An effective MIS provides decision makers with accurate and timely information 
that improves the information base from which decisions are made. Statewide 
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network decision makers may want to consider the creation of an MIS to capture, 
organize, analyze, compare, and report a range of data describing network activities 
and services. Unfortunately, many network evaluators have yet to formalize some 
type of MIS for their network and are thus unable to: 

• Comprehensively capture, organize, analyze, and report information that 
describes network activities and services, network-related costs and 
expenditures, users of the network, and other institutional information related 
to the network. An MIS can 

- Identify and respond to network service strengths and weaknesses or the 
needs of network users, 

- Demonstrate the overall effectiveness and efficiency of the network, track 
various service or use trends, and report such information on a timely 
basis, 

- Simplify and standardize network data gathering and reporting procedures 
across the network, and 

Monitor ongoing network activities and project network costs effectively. 

Perhaps most importantly, statewide networks without some type of MIS are unable 
to determine which specific types of data are most important for collection and 
analysis. As such, data collection activities to monitor and assess the network are 
haphazard, uncoordinated, and likely to carry across the network depending on who 
collects the data. 

Precision of Measurement 

The rapidly changing electronic network technology, coupled with the multi- 
dimensional aspects of electronic networks, dramatically affects the ability of 
network evaluators to precisely measure various components of the network. 
Network performance measures are, therefore, perhaps best viewed as estimates of 
network performance, rather than precise performance measures. The accuracy of 
network measures are affected by such issues as the: 

• Technological aspects of the statewide network. For example, depending how 
users of the network establish network connections, system administrators 
cannot track individual user movement throughout the network (an issue 
created by "guest" or "anonymous" logins). 

• Ability of evaluators to identify, define, and measure network performance 
variables. Network evaluators report to multiple stakeholder groups, all of 
whom have different expectations of network services, view the purpose of 
network evaluation differently, and envision different outcomes from network 
evaluation activities. In such an environment, it is at best difficult to agree on 
variables to measure and ways in which to measure such variables. 

• Difficulty in knowing what network evaluators are counting (Piper, 1996 - See 
Appendix D). There are multiple aspects of network traffic that can be 
counted in some fashion, for example, users, hits on a homepage, and 
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movement throughout a web site. One connection may not, however, equate to 
one user -- one connection in a library may be maintained during normal 
business hours, but several dozen users may use that connection. 

• Use of standardized procedures to collect network performance data. The 
determination of overall network performance and impact in the statewide 
network environment relies on the use of common data collection elements and 
techniques throughout the network. This requires that all the distributed 
components of the network collect, or have the capability to collect, the same 
data in the same manner. 

The electronic networked environment is, therefore, fluid and difficult to precisely 
measure. Evaluators can, however, determine variables to measure, ways in which 
to measure those variables, and use appropriate data collection techniques that allow 
for meaningful interpretation of such variables. These measures are, though, 
acceptable estimates of network performance rather than precise network 
measures. 

Inventing New Methods and Instruments 

To some extent, networked services and programs oftentimes require new 
methods for data collection and innovative techniques for designing data collection 
instruments. For example, a method was developed for this project to compare and 
assess a number of state library web sites. Criteria for assessing these sites had to 
be developed, and then they were measured through a "script" of procedures to insure 
reliable and valid data collection (see the final report Compendium). Thus, the idea of 
"assessing a web site" entailed a significant level of rethinking of method and 
instruments. 

"Pop-up" questionnaires on web pages are another example of new methods and 
instruments which we are only beginning to test and understand as data collection 
instruments. The idea of having a short questionnaire "pop-up" on selected web 
pages and determining factors that contribute to or detract from their completion are 
not, as yet, well understood. And determining the degree to which respondents to the 
“pop-ups” are representative of all users of the webpage offers new challenges for 
assessing the generalizability of those responses. 

When new methods and instruments are used, there is some responsibility on the 
part of the investigators to assess the appropriateness, usefulness, reliability, and 
validity of the techniques. When the researchers developed and used a web 
assessment script comparing Sailor to other state-based web sites, the script also 
collected data about the users' assessment of that survey as a data collection 
technique. Such an approach promotes the ongoing development and improvement of 
new data collection techniques. 

Obtaining Webmeister Cooperation 

Another key issue that must be addressed in completing network-based 
evaluations is obtaining cooperation from the network staff and webmeisters. These 
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staff, typically, are exceptionally knowledgeable about the configuration of the 
network, the points at which "sniffers" should or can be located, and how best to 
install and manage such devices. To a large degree, the amount of evaluation that 
researchers can successfully accomplish is dependent on the degree to which the 
network staff can, or will, work with the researchers. 

Oftentimes, the cooperation needed also has to do with the costs for implementing 
assessment techniques. Sniffer software, programming expenses, other equipment, 
etc., can affect the degree to which the network staff are willing and able to assist in 
the assessment. For very large, statewide networks, implementing sniffers at all 
routers can be both expensive and can negatively affect network performance. Such 
issues affect the overall options available to complete the evaluation. 

Defining the Sailor Network 

A key challenge facing the study team was defining just what exactly the Sailor 
network is. Rather than being a single entity, the Sailor network is the sum total of 
many parts. For this study, however, we define the Sailor network to include the: 

• Bhysical Infras tructure that consists of the network backbone, routers, 
servers -- in essence the hardware and software that are minimally required to 
provide electronic network services. 

* Network Services that consist of information-based (e.g., on-line license 
applications, electronic tax filing) and database services (e.g., on-line 
encyclopedias, inter-library loan requests) provided via the network. 

* Content Services that consist of unique information resources (e.g., historical 
state documents and images, state government legislation, local government 
information) available over the network. 

* Governance Structures that develop and oversee the policies and procedures to 
direct the network’s development over time and on a day-to-day basis. Further 
complicating the issue of governance is that governance structures can exist in 
the service and infrastructure components identified above. 

The Sailor network is, therefore, a multi-dimensional and complex entity. 

Contending with the Measurement Issues 

The study team controlled for such measurement and definition challenges as best 
as possible by: 

• Conducting pre-tests of all data collection activity instr um ents 

• Seeking input from the Advisory Committee on data collection instruments 
and techniques 
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• Testing software and hardware to be used in data collection activities prior to 
use 

• Eliciting the cooperation of data collection participants prior to commencing 
data collection activities 

• Applying an evolutionary methodology that allowed previous data collection 
activities to inform future data collection activities 

• Conducting post-data collection activity debriefings with data collection 
activity participants to identify issues/concems about collected data. 

These actions served to provide appropriate, valid, and reliable data for this study. 

To form a baseline of fundamental Sailor network operations, in terms of 
backbone, content, and services development, the study team reviewed initial and on- 
going Sailor network planning documentation. The following section presents selected 
Sailor planning activities throughout the Sailor network’s development. 



THE BEGINNING OF THE SAILOR NETWORK 

Sailor network planning 3 was initially begun at the request of the Maryland State 
Library Network Coordinating Council. The first broad overview of the project was 
prepared by the Seymour Working Group in December 1992. The Seymour Plan can 
be divided into five components: 

• Definition. The original intent of Seymour was to provide an electronic 
infrastructure to connect Maryland libraries and residents. All Marylanders 
could conceivably access a statewide system of data and resources via the 
Maryland library system electronically. 

The electronic infrastructure of Seymour provided multiple levels of activity. 
These included: 

- Provision of a telecommunication backbone 

- Fax capabilities, electronic mail, and bulletin boards 

- Resource sharing to include full text data bases, bibliographic information, 
and other electronic resources 

- Freenets and community resource/information systems 

- Internet access. 

• Vision. Seymour generated their vision statement in 1989 (DLDS, 1989). The 
mission statement reads: 



3 Although the name Sailor is used throughout this report, initial planning for 
Maryland’s network occurred under the name of Seymour. 
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The Maryland State Library Network will provide the residents of 
Maryland with rapid, easy access to information, materials, and 
services from any available information source. 

Subsequently, Seymour's vision statement called for easy access to 
information, materials, and services. This access would be available to all 
Maryland residents from home, office, school, or library (See Appendix A). 

• Overall Objectives. In April 1992, the Seymour Working Group developed four 
objectives designed to make Seymour's vision statement a reality. They were: 

1. June 30, 1994 - Telecommunication backbone existent with 79% of 

Maryland users provided access to Seymour via a local telephone call. 

2. June 30, 1994 - Intemet/NREN Doorway available. 

3. June 30, 1995 - Dial access capability for 96% of all Maryland libraries. 
The access services of Find a Book, Find an Article, Find a Fact, and 
Find an Agency available. 

4. June 30, 1998 Seymour services fully implemented, with Find 
Government Information available. 

• Implementation. The Seymour Plan called for an incremental development and 
implementation approach, with telecommunications labeled as the first 
component. Implementation was planned to begin June 1, 1993, with f ull 
operation by July 1, 1994. The plan acknowledged the need for a development 
plan for maintenance and "help" services and stated, "The requirement for this 
service and potential supplier will be identified as the plan is developed (The 
Seymour Plan, 1993, p. 4). 

The implementation component had essentially four telecommunications- 
related categories: 

Operations. The Seymour Plan called for reliance on all appropriate 
national standards, such as Z39.50 (computer interfacing), Z39.58 
(common command language), US MARC, and ISO 1016000/10161 
(interlibrary loan). Also noted was a “spoke and hub” arrangement, with 
the hubs denoted as UMCP, Pratt, UM Eastern Shore, and UMAB. These 
hubs are connected by the University of Maryland Network or an Internet 
connection. Telecommunications implementation was to begin with the 
existing nodes at Prate and UMCP. It was suggested that 
telecommunications may be best handled by UMCP with a possible role for 
the State Library Resource Center (Pratt) 

Mention was also made of the need for “user support specialists” and listed 
the Regional Library Resource Centers (Hagerstown, Charlotte Hall, and 
Salisbury) and Pratt as the primary providers. The plan stated that these 
centers were *. . ."well positioned to be the primary providers of user support 
and backup services (p. 11). 
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- Backbone . The backbone was designed to link major libraries in Maryland 
via the TCP/IP protocol. The plan projected local phone access to the 72% 
of residents living in 19 of Maryland's 24 local jurisdictions (counties + 
Baltimore City). The initial implementation for this backbone was to begin 
in late FY 1993, with a projected completion date of June 30, 1994. 

- Library Infrastructure . Five remaining counties were found to lack the 
appropriate equipment for the initial Seymour backbone connection. 
Integration of these counties into the backbone was addressed under 
Project LINKUP (The Seymour Plan, 1993). The intent was to place 
switching and terminal server equipment and modems strategically around 
the state. For example, the rural counties of Caroline and Dorchester were 
targeted with projected resident access of 100% and 89%, respectively. In 
addition, LINKUP proposed extended coverage to three of the initial 
backbone jurisdictions (Garrett, Montgomery, and St. Mary's). Project 
LINKUP was to begin in FY 1994, and the completion date was slated for 
June 30, 1995 (see Figures 3 and 4). 

- Education . Throughout the Seymour Plan emphasis was placed on the 
educational component of the system. However, education as it related to 
vendors and providers was absent, as DLDS operated under the 
assumption that the vendors and providers possessed adequate technical 
expertise to assist DLDS develop the Sailor backbone. This assumption 
proved erroneous, and a co-educational effort between DLDS, Sailor 
Operations, and Bell Atlantic ensued throughout the development of Sailor. 

• Resource I Information Services. These services were designed to build upon 
existing Maryland bases of automation and access. Two facets of 
resource/information development and planning were identified: 

- Migration . The Seymour Plan identified "clusters" of libraries that used 
automated systems. For example, the CARL system served the 
University of Maryland (13 sites) and 1 public library, and the CLSI 
system served 11 public library systems. In addition, these clusters were 
viewed as evolving due to new system development and installation. Most 
clusters had the potential to provide a single type of interface between 
Seymour and local automated systems. For sites without a single type of 
interface (eg. CLSI), a Z39.50 interface that standardizes communications 
could be installed. Through dial access or a Z39.50 interface, it was stated 
that all Maryland libraries have the potential to connect with Seymour. 

- Retrieval Services . Retrieval services included a number of information 
content- and service-related projects. Such projects included: 

- Project Interlibrary Loan. Solicitation to vendors via a request for 
information (RFI) for resource sharing software to meet the needs of the 
system. Planned RFI distribution - January, 1993, and 
recommendations to be made by September 1993. 
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Figure 3. Sailor Network Project Linkup. 
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Figure 4. Sailor Network ISDN Topology. 






17 



Bertot and McClure 



Sailor Network Assessment Final Report 



September 1. 1996 



- Project Seymour Database Management System. A plan for 
maintaining Seymour locally for libraries that lack automated system 
platforms. 

Project USING INTERNET VIA SEYMOUR. This project envisioned 
the development and promotion of Internet access to Maryland libraries. 
Provision of information packets to Directors were to be delivered by 
April 30, 1993 and training packets by late spring 1993. Training was 
to continue through the next fiscal year. 

- Project FIND A BOOK. Provision of electronic access to Maryland 
bibliographic records and ILL requests. Initial implementation was 
slated for FY 1994, with increasing responsibilities and management of 
ILL from FY 1995 through FY 1997. 

Project FIND A FACT. A two-stage plan designed to answer specific 
questions or retrieve information on a particular topic. Stage 1 - ability 
to search through encyclopedias, almanacs, etc. to begin FY 1994. 
Stage 2 - enable a FETCH! command for data from all databases. 
Development of FETCH! FY 1995-98. 

• Funding and Governance. The issues related to governance are unclear in the 
Seymour Plan (see Figures 5 through 9). Nevertheless, it was determined that 
governance would be addressed by The Network Coordinating Council. 
Governance issues included: 

- Network policies and procedures 

- Development issues 

- Future possibilities, decision- makin g 

- Definition & oversight of access to government databases 

The rapidly changing Internet technology arena, combined with an overall lack 
of a clear formal Sailor management structure, set into motion a flexible, and 
sometimes confusing and competing, Sailor network management framework 
(as Figures 5-9 demonstrate). 

Funding for initial Sailor network planning, development, and implementation 
came primarily from federal Library Services and Construction Act (LSCA) 
monies (see Appendix B). The individual library systems also contributed a 
significant amount of in-kind funding for the Sailor network (e.g., personnel 
time), but such contributions were difficult to assess. From federal 1993-1995 
Fiscal Years, a total of $2,062,773 was diverted from Maryland public libraries 
to operate Sailor. Beginning with this year, state FY1996, Maryland 
appropriated $250,000 to EPFL for maintaining the Sailor network backbone. 

The above discussion of Sailor planning activities serves as a basis for measuring 
the attainment of Sailor network objectives, interpreting the findings from the data 
collection activities, and recommending future directions for Sailor network 
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Figure 5. Sailor Network Organizational Hierarchy. 
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Figure 6. Sailor Network Management Entities. 
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Figure 7. Sailor Network/Technical Management Structure. 
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Figure 8. Sailor Network Service Entities. 
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Figure 9. Sailor Network Content Development Entities. 





Appoint the Tropical Area 
Reviewers 

Development of Sailor Web 
site development policies 




Bertot and McClure 



Sailor Network Assessment Final Report 



September 1. 1996 



development. The next section presents Sailor network usage statistics gathered 
during the evaluation project. 



SAILOR NETWORK USAGE STATISTICS 

As part of this study, the researchers, with assistance from the Sailor Operations 
Center staff, collected both Sailor network server usage statistics and case site 
Ascend router usage statistics.4 The initial methodology intended to simultaneously 
collect case site and server statistics throughout March 1996. As this was the first 
attempt at Ascend router usage collection, however, technical difficulties prevented 
the collection of Ascend router statistics during March 1996. This section, therefore, 
presents Ascend router and Sailor network server statistics from two differing time 
periods - March 1996 for the Sailor network and May 20, 1996 through July 25, 1996 
for case site Ascend routers. 

The types of statistics gathered during this study include the following: 

• Sailor network server sessions initiated by IP (Internet Protocol) address to 
determine from where server traffic is initiated (see Figure 10); 

• Sailor network server sessions by day of week (see Figure 11); 

• Sailor network server sessions by hour of the day (see Figure 12); 

• Sailor network server most frequent file requests (see Figure 13); 

• Sailor network server requests to Maryland state agency Web sites (see Figure 



• Sailor network Help Desk usage statistics for March 1996 (see Figures 14 and 



• Case site Ascend router usage by hour (see Figures 16-19); 

• Case site Ascend router traffic by day (see Figure 20); and, 

• Case site Ascend router traffic call duration (see Figure 20). 

The researchers note in the Compendium that other usage statistics can and should 
be collected by Sailor network-connected entities. 

During this study, no Maryland library that connected to the Sailor network 
through a DLDS-supported connection did so through a log-on procedure. As such, 
users of Sailor connected to the sever as “guest” (as indicated in Figure 10 through 



4During this study, the Sailor network operators were in the process of 
terminating the Sailor gopher server, and did so on June 30, 1996. The declining 
gopher statistics, therefore, reflect the impending gopher server shutdown. 
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the “guest” designation). Furthermore, a majority of Maryland libraries did not have 
a Point-to-Point (PPP) type of connection to the Sailor network. The basic Sailor 
network connection, therefore, consisted of a non-graphical “Lynx” or gopher 
connection to the Sailor home page or gopher server (again included in the “guest” 
category in Figure 11). Some Maryland library systems, however, established 
separate PPP connections. These are reflected in the “Web” category in Figure 10. 

For March 1996, the Sailor Web server received a total of 592,979 hits and the 
Sailor Gopher server received a total of 285,622 hits (see Figure 10). Of those 
592,979 Web hits, 396,497 were “guest” logins. Of the 285,622 Gopher hits, 243,790 
were “guest” logins. As Figure 10 demonstrates, a majority of the Sailor network 
“guest” logins originate from Maryland public libraries, with 95.0% of all “guest” 
logins. The top five public library system users of the Sailor servers within the 
Maryland public library community for March 1996 were Montgomery County 
(16.96%), followed by Enoch Pratt Free Library (13.10%), Harford County (8.24%), 
Anne Arundel County (8.19%), and Prince George’s County (7.23%). The top three 
non-”guest” login users of the Sailor Web server are .com (commercial) users (9.30%), 
followed by .net (network provider-most likely users that purchase Internet accounts 
through Internet service providers) users (4.60%), and .edu (academic institution) 
users (3.62%). 

In general, Sailor server traffic is steady throughout the week (see Figure 11). The 
slowest Sailor network use day in March was Sunday, with 105,376 total Gopher and 
Web hits, while the most heavy network traffic day in March was Friday, with 
144,223 total Gopher and Web hits. The Sailor Gopher and Web server use peaked 
between the hours of 2:00PM and 4:00PM, with 128,109 hits (see Figure 12). It is 
interesting to note that Sailor server traffic was steady (range of hits from 65,923 to 
128,108) between 8:00AM and midnight each day, while the least amount of server 
traffic was generated between midnight and 8:00AM each day (range of hits from 
10,688 to 30,987). 

Not surprisingly, the most heavily accessed file on the Web server during March 
1996 was the Sailor home page with 169,831 hits (28.66%), as each Web session 
generally begins at the home page (see Figure 13). In terms of content-oriented Sailor 
files, the top five most accessed files were the Topics (e.g., Arts & Humanities, 
Business & Consumer) pages with 103,053 (17.38%) hits, followed by the Forms (e.g., 
Lycos and Webcrawler) searches pages with 70,881 (11.95%) hits, the Picture file 
(e.g., .gif images) pages with 42,165 (7.11%) hits, the About Sailor (e.g., Sailor project 
information) pages with 29,194 (4.92%) hits, and the Documents (e.g., miscellaneous 
documents, e-mail gateway) pages with 28,471 (4.80%) hits. Figure 13 demonstrates 
that the Maryland state agency pages were not heavily used during March 1996, 
comprising only 13.10% of the Sailor Web server traffic. As Figure also 14 indicates, 
the Sailor Gopher was predominantly used for external Internet connections, with 
64,316 (22.52%) hits. 

During March 1996, the Sailor Operations Center (SOC) Help Desk responded to 
2,298 help requests (see Figure 14). Of those requests, 562 (see Figure 14, Staff 5 
and Staff 6) - 18.7% - were received via e-mail. In addition, 365 Help Desk requests 
- 12.2% - were from Metnet (Maryland’s educational K-12 network) users. The SOC 
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Figure 10. Sailor Server Traffic for the Month of March 1996 by IP Address. 
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Figure 10. Sailor Server Traffic for the Month of March 1996 by IP Address. 
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0.02% 


24 


0.01% 


121 


0.02% 


•uk 




22 


0.01% 


73 


0.03% 


277 


0.05% 


other 

non-us 




30 


0.02% 


464 


0.16% 


1,831 


0.31% 




SUBTOTAL 


460 


0.27% 


1,085 


0.38% 


2,849 


0.48% 


unresolved IPs and failed 
requests 


2,321 


1.38% 


10,170 


3.56% 


45,431 


7.66% 


















TOTALS* ** 




167,913 


100.00% 


285,622 


100.00% 


592,979 


100.00% 



l The “guest” logins equate to the number of users that accessed the Sailor network servers as 
guests. In this figure, 167,913 guest logins generated 243,790 gopher and 396,497 Web network 
server hits, respectively (see the “Sailor” row in the figure). The IP addresses in the “guest” column 
indicate the addresses that generated the guest login — e.g., the Baltimore County library system 
users generated 9,494 guest logins to the Sailor network servers. 

2 The “Gopher” column is the number of gopher server hits generated by users of the Sailor network. 
Of the 285,622 total hits, 243,790 were generated by “guest” logins. The remaining hits were 
generated by library system local area networks or OPACS that could directly access the Sailor 
gopher and other non- Ascend router dial-up users. 

^The “Web” column is the number of Web server hits generated by users of the Sailor network. Of 
the 592,979 total hits, 396,497 were generated by “guest” logins. The remaining hits were 
generated by library system local area networks or OPACS that could directly access the Sailor Web 
page and other non- Ascend router dial-up users. 

*Site not open to public during March 1996. 

**Totals do not sum to 100.00% in all cases due to rounding errors as well as missing data points. 
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Figure 11. Sailor Network Server Use by Day of Week for March 1996. 
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Figure 12. Sailor Network Server Use by Hour for March 1996. 
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Figure 13. Sailor Server File Usage for March 1996. 



Directory 


Hits 


% 


Comments 


WEB 








root 


169,831 


28.64% 


Home page 


sailor 


29,194 


4.92% 


About sailor project 


md 


7,583 


1.28% 


Maryland government page 


mdcounty 


9,310 


1.57% 


Maryland county pages 


mdlibs 


7,738 


1.30% 


Maryland libraries 


mdedu 


3,149 


0.53% 


Maryland education 


mgiug 


2,884 


0.49% 


State government Internet Users Group 


topics 


103,053 


17.38% 


Topic homepages 


forms 


70,881 


11.95% 


Search forms 


gifs 


42,165 


7.11% 


Picture files 


docs 


28,471 


4.80% 


Misc. documents, includes e-mail gateway page 


cgi 


22,813 


3.85% 


Scripts and searches 


lynx 


12,524 


2.11% 


GPO: 1785 
MEC imagmap: 6623 
MEC searches: 1448 
Full-text Sailor: 1343 
URL Sailor: 8615 
State phone dir: 824 
Minority biz dir: 528 
Maryland state docs bib: 457 


stats 


2,069 


0.35% 




archives 


1,132 


0.19% 


Old "What's New, Hon** 


gpo 


429 


0.07% 


Government Printing Office help file 


SUBTOTAL 


615,367 


86.9% 




MD AGENCIES 








mec 


65,717 




Maryland Electronic Capital 


dllr 


5,536 




Dept of Labor, Licensing, and Regulation 


msla 


2,364 




Lottery 


dmil 


1,133 




Military 


mda 


968 




Agriculture 


msde 


930 




Education 


sra 


341 




Retirement 


mgcos 


187 




Governors Commission on Service 


msbca 


101 




Board of Contract Appeals 



0 
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Figure 13. Sailor Server File Usage for March 1996. 



Directory 


Hits 


% 


Comments 


bpw 


100 




Pubic Works 


oah 


89 




Administrative Hearings 


mema 


51 




Emergency Management 


pscp 


48 




School Construction 


osp 


20 




State Prosecutor 


wab 


17 




Wine Advisory Board (ag) 


SUBTOTAL 


77,602 


13.1% 




TOTAL 


592,969 


100.0% 




GOPHER 








root 


70,819 


24.79 % 


Homepage 


internet 


64,316 


22 52% 


External connections 


findlnfo 


33,369 


11.68 % 


By topic 


govinfo 


12,917 


4.52% 


Government information 


looklibs 


12,463 


4.36% 


Libraries 


about 8 ail or 


6,946 


2.43% 


Information file on Sailor 


comminfo 


2,661 


0.93% 


Community information 


whats 


1,037 


0.36% 


Whats new 


TOTAL 


285,622 


71.61%* 





•Totals do not sum to 100.00% to rounding errors as well as missing data points. 
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Figure 14. SOC Sailor Help Desk Log for March 1996. 





Sailor 


Pratt 


Metnet 


Other 


Total 


Staff 1 


178 


84 


60 


401 


723 


Staff 2 


241 


88 


246 


122 


697 


Staff 3 


186 


148 


28 


12 


374 


Staff 4 


226 


76 


14 


10 


326 


Staff 5 


0 


81 


0 


0 


81 


Staff 6 


481 


0 


0 


0 


481 


Staff 7 


98 


52 


10 


31 


191 


Staff 8 


74 


34 


7 


10 


125 


Total 


1484 


563 


365 


586 


2998 



Notes on the above data: 



The number of calls in the category Other for staff 1 is large because that person was responsible 
for retrieving voice-mail and distributing messages to other staff (especially voice-mail that 
accumulated overnight). 

Staff 5 corresponds to the flow of e-mail to a Pratt Accounts Administration e-mail address. 

Staff 6 corresponds to the flow of e-mail to sailor@mail.pratt.lib.md.us (sailor@epfl2). There are 
two sources of mail flow sailor@mail.pratt: Feedback, Please (on sailor.lib.md.us) and mail posted 
directed to that address from the senders e-mail account. 
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Figure 15. Help Desk Calls by County for March 1996.* 



County 


Logged Calls 


Percentage of Total Calls 


Anne Arundal 


39 


7.3% 


Baltimore City 


29 


5.4% 


Baltimore County 


84 


15.7% 


Calvert County 


4 


0.7% 


Carroll County 


15 


2.8% 


Cecil County 


8 


1.5% 


Charles County 


1 


0.2% 


Dorchester County 


6 


1.1% 


Frederick County 


12 


2.2% 


Garrett County 


7 


1.3% 


Harford County 


31 


5.8% 


Howard County 


40 


7.5% 


Kent County 


18 


3.4% 


Montgomery County 


35 


6.5% 


Enoch Pratt 


82 


15.3% 


Prince George’s County 


37 


6.9% 


Queen Anne's County 


2 


0.4% 


Somerset County 


1 


0.2% 


Talbot County 


14 


2.6% 


Washington County 


12 


2.2% 


Wicomico County 


7 


1.3% 


Worcester County 


1 


0.2% 


Other 


49 


9.2% 


Total 


535 


100.0% 



*This figure is best viewed as a rough estimate of Help Desk use by County. Not all 
incoming calls to the SOC Help Desk are tracked by location. 
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